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Abstract. — If Go is a real form of a complex semisimple Lie group G and Z is 
compact G-homogeneous projective algebraic manifold, then Gq has orJy finitely 
many orbits on Z. Complex analytic properties of open Go-orbits D (flag do- 
mains) are studied. Schubert incidence-geometry is used to prove the Kobayashi 
hjrperbolicity of certain cycle space components Cq{D). Using the hyperbolic- 
ity of Cq{D) and analyzing the action of Auto(D) on it, an exact description of 
Autci(D) is given. It is shown that, except in the easily understood case where D 
is holomorphically convex with a nontrivial Remmert reduction, it is a Lie group 
acting smoothly as a group of holomorphic transformations on D. With very few 
exceptions it is just Gq. 



1. Introduction 

Throughout this paper Z denotes a compact projective algebraic manifold 
which is homogeneous with respect to a holomorphic action of a connected 
complex semi-simple group G. Such manifolds are alternatively described by 
Z = G/Q where Q is a (complex) parabolic subgroup of G. A Lie subgroup 
Go of G is said to be a real form of G whenever its Lie algebra qq is defined as 
the fixed point space of an antilinear involution t : g ^ g. Since Gq has only 
finitely many orbits in Z ([WJ), it has at least one and usually many open orbits 
D. Manifolds of the type Z are called flag manifolds and the open Go-orbits D 
are called flag domains. In all applications standard decompositions allow us 
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to reduce to the case where Go is simple and therefore we assume this. Note 
that if Go is complex, being embedded in G by an antiholomorphic map, then 
G = Go X Go. This is the only situation where it may not be assumed that G is 
simple. 

It is possible that D = Z. This occurs when Go is compact and there are sev- 
eral exceptional noncompact cases as well ( Il01il02 l, see Proposition 5.2.1 in 
JFHWJ). From the point of view of the present work, these cases are not of 
interest. Thus we not only assume that Go is a simple noncompact real form of 
G but also that D is not compact. 

Let us fix a maximal compact subgroup Kq of Go and denote by K its complexi- 
fication in G. A basic first example of Matsuki-duality states that Kq has exactly 
one orbit Co in D which is a complex submanifold (IWl). It is in fact the only 
K-orbit in Z which is contained in D. 

Fix q := dimc(Co) and recall that a t^-cycle in Z is a formal linear combination 

C = niXi + . . . + nmXn, 

where the coefficients are positive integers and the Xf are compact, irre- 
ducible, (^-dimensional subvarieties of Z. The set of all such cycles has the nat- 
ural structure of complex space (IE]], see [FHW[ §7.4 for the properties needed 
here). In the particular case where Z is projective, it is the Chow variety. 
Its components are themselves projective algebraic and the G-action on these 
components is algebraic. In many situations we regard Co as the cycle l.Co in 
the cycle space and refer to it as the base cycle. Since the cycle space is smooth 
at Co (^ [FHW[ Theorem 18.6.1), the notion of the irreducible component of the 
cycle space of Z which contains Co makes sense. This is denoted by Cq{Z) and 

Cq{D) := {CgQ(Z);CcD}. 

It should be remarked that, although it is known that Cq{D) is smooth at the 
base point Co, it is an open question as to whether or not it is everywhere 
smooth. 

The following is the first main result of this paper. 

Theorem 1.1. — The complex space Cq{D) is Kobayashi-hyperbolic. 

The proof of this result is established using the Schubert incidence geometry de- 
fined by special Borel subgroups of G. For this, recall that a Borel subgroup B 
has only finitely many orbits O in Z. Each is abstractly algebraically isomor- 
phic to some C'"*^^). These are referred to as Schubert cells. Since the B-action 
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on Z is algebraic, every such orbit is Zariski open in its closure. We denote 
such a closure by S and write S = OUE, where E is the union of the (lower- 
dimensional) B-orbits on the boundary of O. The varieties S, which in fact 
often have singularities along E, are called Schubert varieties. Observe that the 
CW-decomposition of Z which is defined by the Schubert varieties only has 
cells in even (real) dimensions. Thus they freely generate the integral homol- 
ogy H,(Z,Z). 

If Go = KqAqNq is an Iwasawa-decomposition and the Borel group B contains 
the solvable subgroup AqNq, we refer to it as being an Iwasawa-Borel sub- 
group. These subgroups can be viewed in the flag manifold G/B of all Borel 
subgroups as the points on the (unique) closed Go-orbit. 

Now consider the set S{B) of (n-q)-dimensional Schubert varieties of an 
Iwasawa-Borel subgroup B. It follows from Poincare duality that at least some 
of the S in <S(B) have nonempty (set- theoretic) intersection with Co. The fol- 
lowing is the essential ingredient for a number of considerations in this context 
(see Theorem 9.1.1 in [FHWJ). 

Theorem 1.2. — If S = OOE G S{B) is an (n-q)-dim.ensional Schubert variety 
of an Iwasawa-Borel subgroup of G and S n Co 7^ 0, then S n Co is a finite subset 
{z\, . . . ,Zm} which is contained in O. At each z, the intersection S n Co is transversal. 
The orbit AqNq.z, =: E, is open in S and closed in D; in particular S n D C O. 
Furthermore, L, n Co = {z/}. 

If S is as in the above Theorem, then we consider the incidence variety 

Ds := {CeCc{Z);CnE^(Z)}. 

It is important that, for appropriate multiplicities of its components, this is a 
Cartier divisor, (see | |FHW |, §7.4, and the Appendix of flHSl ). Theorem 11.11 is 
proved by considering the maps given by the linear systems defined by the in- 
cidence divisors D5. These are pieced together in such a way that after paying 
the small price of a finite map, C^(D) is realized as an open subset of a hyper- 
bolic domain in a product of projective spaces. The hyperbolicity of this latter 
domain is proved by applying the classically proven fact that the complement 
the union of 2n + 1 hyperplanes in general position in P„ is hyperbolic. 

The method of Schubert incidence geometry has played a role in much of our 
work over the last years. In particular, in [HlJ we used this method to prove 
the hyperbolicity of the group-theoretically defined cycle space Aio which is 
the connected component containing the base cycle of the intersection G.Co n 
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Cq{D). The proof here of hyperbolicity in the case of the full cycle space Cq{D) 
is substantially more involved. 

It should be remarked that, except for a certain Hermitian cases which are well- 
understood, Md only depends on Go, i.e., it doesn't vary as the G-flag mani- 
fold Z and the Go-flag domain D vary (see tFHW] ). On the other hand, Ccj{D) 
varies (sometimes wildly) as Z and D vary (Part IV of |!FHW[ V Thus it would 
seem that the cycle spaces Cq{D) might provide a wide range of interesting 
Kobayashi-hyperbolic spaces which could be useful for holomorphically real- 
izing Go-representations. 

Theorem 11.11 is one of the essential ingredients in the proof of the following 
second main result of this paper (see ^ in particular Theorem|5.1|). 

Theorem 1.3. — Unless D is holomorphically convex with nontrivial Remmert re- 
duction, the automorphism group Auto{D) is a Lie group acting smoothly on D as a 
group of holomorphic transformations. 

This leads to a detailed description of the connected component Auto{D)° for 
every flag domain D. In order to prove Theorem 11.31 we characterize holo- 
morphic convexity by a cycle condition and show that if this condition is not 
satisfied, then the natural map Auto{D) — > Auto{Cq{D)) is finite-fibered and 
is a local homeomorphism onto its closed image. The desired results then fol- 
low from the fact that the automorphism group of a hyperbolic complex space 
is a Lie group. 

As is indicated in the statment of Theorem 11.31 the exception to the finite- 
dimensionality occurs in a setting which is optimal from the point of view 
of complex analysis, namely where D is holomorphically convex but not Stein. 
This can only occur in the case where Go is of Hermitian type and even in 
that case it is very rare. The Remmert reduction D — > D, which in this case 
is induced from a fibration Z ^ Z of the ambient flag manifolds, is a Go- 
equivariant homogeneous fibration onto the Go-Hermitian symmetric space of 
noncompact type embedded in its compact dual. This situation can be charac- 
terized in a number of ways (see 

In §6 it is shown that unless Z = G/Q and Q is a maximal parabolic subgroup, 
or equivalently ^2(2) = 1, it follows that Go := Auto{D) = Gq. Even when 
b2{Z) = 1 there are very few exceptional cases. A key point for the classifica- 
tion of these cases is that if Gq properly contains Go, then the complexification 
G also acts transitively on Z. Therefore we are a position to apply Onishchik's 
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classification of that situation, i.e., that where two different complex simple 
groups act transitively on Z. 



2. Hyperbolicity of the cycle space 

This section is devoted to the proof of Theorem ll.il For this we at first consider 
an Iwasawa-decomposition Go = KqAqNq and fix an Iwasawa-Borel subgroup 
B which contains AqNq. Let ScgiB) be the set of (n-q)-dimensional B-Schubert 
varieties which have nonempty intersection with the base cycle Cq. For S = 
Cue G Scq (B) let D be a Cartier divisor with support on the incidence variety 
defined by E and let L be the associated line bundle on Cq{Z) with section s 
defining D. 

We may choose G to be simply-connected so that L has the unique struc- 
ture of a G-bundle with s being a B-eigenvector in the finite-dimensional G- 
representation space T{Cq{Z),L). Since this representation may not be irre- 
ducible, we let V be the irreducible subrepresentation which contains s as a 
B-highest weight vector and consider the G-equivariant meromorphic map 

defined by ^(x) := {s;s(x) =0}. 

Observe that since E is contained in the complement of D, the restriction of cp to 
Cq{D) is base point free. Furthermore, the image (p{Ccj{D)) is contained in the 
complement P(V*) \ Hq of the hyperplane which corresponds to the section s. 

Note that since Hq is AoNo-invariant, the family T := {gHo}^eGo is compact. 
As a consequence the union UheT^ is compact and its open complement O 
contains the image (p{Cq{D)). In [Hi] (see also | |FHIIFHW| ) we utilize the fact 
that the d-dimensional G-representation on V* is irreducible to prove the fol- 
lowing fact. 

Theorem 2.1. — The family T contains 2d — 1 hyperplanes Hi,. . . ,H2d-\ which 
are in general position inW{V*). In particular, O is Kobayashi-hyperbolic. 



The technical work for the proof of Theorem 11.11 amounts to carrying out the 
above construction for each of the Schubert varieties in <S(B) and then to show 



6 



ALAN HUCKLEBERRY 



that the restriction to Cq{D) of the product of the resulting maps is finite- 
fibered. For a precise statement let Si, ... , Sm be a list of the Schubert vari- 
eties in Scg{B) and Vi, . . . ,Vmhe the irreducible G-representations with high- 
est weight vectors Si, . . . , Sm obtained as above. Let cpj : Cq{Z) — > F{V*) be the 
associated meromorphic maps. 

Proposition 2.2. — The restriction of the meromorphic map 

xl^:=cpix...x(p,„: C^Z) ^ F{V^*) x . . . x P{V*) 
to Cq{D) is holomorphic and finite-fiber ed. 

The proof of this Proposition, which goes by assuming the contrary and deriv- 
ing a contradiction, requires some notational preliminaries. For this let Cq{Z) 
be the normalized graph of ip and 

{p:q{Z)^P{V,*)x...xP{V,:) 
be the resulting holomorphic map. 

We suppose to the contrary of the claim in Proposition 12.21 that Y is a compact 
complex curve in some i/7-fiber and that Y has nonempty intersection with the 

(biholomorphic) lift of Cq{D) in Cq{Z). Under that assumption let Y denote 
the projection of Y in Cq{Z), Xy the preimage of Y in the the universal family 
X Cq{Z) and let X be the the (q+l)-dimensional image of Xy in Z. 

Now consider the intersection of X with any one of the Schubert varieties in 
<Sco(B) which for simplicity we denote by S = OUE. Since X n D 7^ and 
every such S intersects every cycle (transversally) in D in only finitely many 
points, it follows that S n X contains a 1-dimensional component which has 
nonempty intersection with D. Since O = C^^'-^\ the intersection with X HE 
is nonempty. 

Lemma 2.3. — Every C G Y with C HE ^ & is contained in the base point set 
of the meromorphic map defined by the Schubert variety S. 

Proof. — Let s be the section of the bundle L which is the B-highest weight vec- 
tor in the representation space V defined by the incidence divisor Dg. Since E 
is contained in the complement of D, for those cycles C G Y n Cq{D) it follows 
that s(C) 7^ 0. But s(C) = 0. Thus if C were not a base point, it would follow 
that (p{C) 7^ (p{C). This is contrary to Y being contained in a ^-fiber. □ 
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Proof of Propos ition \2.2\ In order to complete the proof, we will produce a Schu- 
bert variety S G Scg{B) so that if C G Y as in the above Lemma, then C is 
not a base point of the associated meromorphic map. For this we consider the 
closure cl(G.C) of the G-orbit of C in the full cycle space Cq{Z) and let Ci be a B- 
fixed point in a closed G-orbit inbd(G.C). Then Ci is the union of finitely many 
q-dimensional Schubert varieties. Recall that Poincare duality in Z is realized 
by the intersection pairing of B-Schubert varieties with Schubert varieties of 
the opposite Borel subgroup B*. In particular, there is an (n-q)-dimensional 
B*-Schubert variety Si = OOE which has nonempty finite intersection with 
Ci with Ci n E = 0. 

Since convergence at the level of cycles (regarded as points) is the same thing as 
Hausdorff convergence, it follows that if gC is sufficiently near Ci in the cycle 
space, then Si has nonempty finite intersection with gC with gC Ci E likewise 
being empty. Consequently there is a proper algebraic subset A of G with the 
property that if g ^ A, then C has finite nonempty intersection with gSi and 

c n = 0. 

Now Si is not a B-Schubert variety, but if we choose B carefully, the opposite 
Borel B* is also an Iwasawa-Borel subgroup which is Ko'Conjugate to B. For 
the sake of completeness let us explain this choice in some detail. Let cr be 
the involution which defines the maximal compact subalgebra Qu and which 
commutes with the involution t which defines qq so that cr\Qo ='■ Ois the Cartan 
involution. This defines the Cartan decomposition go = © Po with qq defined 
to be a maximal Abelian subspace of po- Let mo be the centralizer of ao in to 
and choose a toral subalbra to in mo so that [)o = to © ^o is a real form of the a- 
invariant Cartan subalgebra f) = t ffi a of g. Finally define a system of positive 
roots for g with respect to f) so that the direct sum xIq of the positive restricted 
root spaces defines our Iwasawa decomposition go = to® ao®nQ. Now let b 
be the Borel subalgebra in the minimal parabolic subalgebra p = m ® a ® n+ 
which consists of i) together with the positive root spaces. This is a semidirect 
sum b = bm x (a ® n"*") where bm is a Borel subalgebra of m. The opposite 
Borel algebra b* which consists of [) together with the negative root spaces is 
of the form b* = bj^ tx (a ® n~) and go = to® ao® xIq is the corresponding 
Iwasawa-decomposition. 

Since Ko acts transitively on the Iwasawa-decompositions with compact sum- 
mand to, for some ko G Ko it follows that koSi = S, where S G ScgiB). 
As a result, for g ^ Afc~^ it follows that C n gS is nonempty and finite, and 
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C n gE = 0. But the section g{s) defined by the incidence divisor of gE is lin- 
early equivalent to that defined by S and CCigE = translates to g{s) (C) 7^ 0. 
Consequently C is not a base point of the meromorphic map cp defined by 
S. □ 

We now come to the final steps in the proof of the hyperbolicity of the cycle 
space. 

Proof of Theorem UTIl Let W be the image of holomorphic map ip : Cq{Z) — > 
P(V^*) X . . . X P(V;;) and tp stein : W ^ W its Stein factorization. Note that 
Proposition 12.21 can be reformulated to state that Q(D) is naturally embedded 
in W. Recall that ¥{V*) contains an open subset Oy which is the complement 
of the union of the hyperplanes in a family which is Kobayashi-hyperbolic 
and which contains the image of the restriction of cpj to Cq{D). Thus the in- 
tersection O of the product Qi x . . . x Clm with W has the same properties: It 
contains the image of Cq{D) (now regarded as a subset of W) and is Kobayashi- 
hyperbolic. To complete the proof observe that the restriction of Ostein to the 
preimage Q of O is a proper finite-fibered holomorphic map. Since Q is hyper- 
bolic, it follows that O is likewise hyperbolic (IKI, Proposition 3.2.11). Finally, 
since Cq{D) has been realized as an open subset of O, it is immediate that it is 
hyperbolic as well. □ 



3. Cycle reduction 

Here we discuss two equivariant holomorphic equivalence relations which are 
defined by the cycles in D. The first has been introduced in [H2], but our 
point of view here is different and for the sake of completeness we provide the 
essential details. 

Cycle connectivity. — Let us say that two points p,q ^ D are connected hy cy- 
cles if there are finitely many cycles C\, . . . , £ Cq{D) so that p G Ci, q G Cjv 
and the chain Ci U . . . U C]v is connected. This notion defines an equivalence 
relation where two points are equivalent whenever they are connected by a 
chain of cycles. It is Go-equivariant in the sense that for all g G Go 

pr-^q ^ g{p)r~.g{q). 

Let po be a base point with the property that Kq-Po = Cq is the base cycle. If p 
is connected to po by a chain Ci U . . . U Cjv to po with po G Ci and p G Cjsi and 
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^0 £ ^0/ then ko{p) is connected to po by the chain Q U ko{Ci) U . . . U ko{Cj^). 
Thus the equivalence class [po] is Ko-invariant. 

Now write D = Gq/Hq where Hq is the isotropy subgroup of the point po- 
The reduction D ^ D/^ defined by cycle connectivity is Go-equivariant and 
therefore it is a homogeneous fibration Gq/Hq Gq/Iq. If Iq = Hq, in other 
words if any two points can be connected by cycles, we say that D is cycle 
connected. 

Proposition 3.1. — IfDis not cycle connected, then Iq — Kq, i.e., the base D of the 
cycle reduction is the associated GQ-Hermitian symmetric space. 

Proof. — Above it was shown that Kq is contained in the stabilizer of the equiv- 
alence class [po]- Recall that Kq is not only a maximal compact subgroup of Go 
but is in fact a maximal subgroup. Thus the stabilizer of [po] is either the full 
group Go or is Kq. In the former case D is cycle connected and in the latter case 
[po] = Co. □ 

Observe that in the case where D is not cycle connected there is an open neigh- 
borhood U of the identity in the complex G-isotropy subgroup Q at pQ which 
stabilizes the equivalence class [pq] = Cq. Since U generates Q, it follows that 
[Pq] is Q-invariant. Define Q as the stabilizer of [pq] in G and note that it con- 
tains both K and Q 

Proposition 3.2. — IfD is not cycle connected, then the restriction R of the fibration 
Z — G/Q^G/Q — Zto the flag domain D is the quotient Gq/Hq = D D = 
Gq/Kq defined by cycle connectivity equivalence in the strong sense that the fibers of 
D ^ D are the same as the fibers ofZ^Z over points ofD. 

Proof. — The restriction R maps D onto a flag domain Di whose Go-isotropy 
at its base point contains Kq. Since D is not cycle connected, the neutral fiber 
of G/Q = Z — )• Z = G/Q is the cycle Co; in particular, Z is not just a point. 
Since the maximal subgroup Kq is the stabilizer of Co in Go, it follows that 
Di = D. □ 

Let us summarize the results of this paragraph. 

Corollary 3.3. — Under the assumption that D is not a Hermitian symmetric space 
ofnoncompact type, the following are equivalent: 
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1. The flag domain D is not cycle connected with reduction Gq/Hq = D — > D = 
Gq/Kq defined by cycle connectivity equivalence. 

2. The flag domain D is holomorphically convex with nontrivial Remmert reduc- 
tion. In particular, the base D of its Remmert reduction Gq/Hq = D — > D = 
Gq/Kq is a Hermitian symmetric space D which is embedded as a GQ-orbit in its 
compact dual Z and its neutral fiber is Cq. 

Proof. — If D is holomorphically convex, then the stated properties concern- 
ing its Remmert reduction are known ([WJ, see also [FHW], p. 63). Thus, in 
order to show that 2.) implies 1.) we must only note that, since its base is Stein, 
the Remmert reduction maps every chain of cycles in D to a point. Therefore D 
is not cycle connected. The direction 1.) implies 2.) follows immediately from 
that fact that the fiber of the reduction defined by cycle connectivity equiva- 
lence is the compact variety Cq and the base is the Stein Hermitian symmetric 
space. □ 



The group Autc)(D) in the holomorphicallyconvex case. — One of the pur- 
poses of this article is to describe in detail the automorphism groups of flag 
domains. The case where D is not cycle connected is special in that Aut0(D) 
is not a Lie group. Let us now describe this group. 

Assume that D is not cycle connected with Remmert reduction D ^ D. Note 
that since the Hermitian symmetric space D is Stein and retractible, the bundle 
D — > D is holomorphically trivial. Thus D is biholomorphically equivalent to 
the product Cq x D of the base cycle and a Hermitian symmetric space of non- 
compact type. The latter can be realized as a bounded domain and therefore its 
automorphism group is a (known) Lie group acting smoothly on D as a group 
of holomorphic transformations. 

Since Aut0(Co) is a complex Lie group, we may consider the space 
Hol(D, Auto (Co)) of holomorphic maps of the base of the Remmert reduction 
to this group. Every cp G Hol(D, Autc)(Co)) defines a holomorphic automor- 
phism of the product Cq x D, namely by {z,w) i— > {(p{w){z),w). Conversely, 
the projection tt : Cq x D ^ D is equivariant with respect to Aut(5(Co x D) 
and its ineffectivity on D is exactly Hol(D, Autc)(Co)). Thus, although it is 
quite large, Aut0(D) can be concretely described. 
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Proposition 3.4. — If D is not cycle connected with cycle equivalence reduction 
given by D ^ D, then 

Auto(D) ^ Hol(D, Auto(Co)) x Auto(l3) . 

The following remark which concludes this paragraph will be of use in the 
next section. 

Proposition 3.5. — If Z — ^ Zi is a G-equivariant fibration which induces a map- 
ping D ^ Di of flag domains and D is cycle connected, then Di is cycle connected. 

Proof. — If Di is not cycle connected, then it is holomorphically convex; in 
particular, 0{Di) ^ C. Thus 0{D) ^ C and as a result D is likewise holomor- 
phically convex (||W[V □ 

Remark. It should be noted that if we had defined the cycle connectivity equiv- 
alence relation using only the cycles in A^D/ i-e., those of the form g{Co) for 
g G G, then the entire discussion above could have been repeated. In particu- 
lar, D is cycle connected in this sense if and only if it is cycle connected in the 
weaker sense where we use cycles from the full cycle space Cq{D). □ 

Cycle separation. — The cycle separation equivalence relation is defined by 
pr^q if and only if every cycle in Cq{D) which contains p also contains q 
and vice versa. In other words, if J-p denotes the family of cycles which 
contain p, then p^^q if and only if = Fq. Since g{J^p) = J^g{p) for all 
g G Auto{D), the equivalence relation is Autc)(D)-equivariant. In particu- 
lar, it is Go-equivariant. Therefore, if D = Gq/Hq, where Hq is the isotropy 
group of the neutral point po in D, then the reduction D — > D/^ =: Disa 
Go-fibration Gq/Hq = D ^ D = Gq/Iq where Iq is the stabilizer of the equiv- 
alence class [po] • 

For p ^ D define Ip := CiceTp^ arid observe that 

[po] = npeXpJp. 
The following is an immediate consequence of this description. 

Proposition 3.6. — The cycle separation equivalence classes [p] are closed compact 
complex analytic subvarieties of D. In particular, if the base cycle Cq is defined by 
Kq-Pq, then [po] is contained in Cq. 
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Using this proposition we now show that reduction by cycle separation can be 
extended to a G-equivariant fibration Z — )■ Z of the ambient projective space. 

Lemma 3.7. — The equivalence class [po] is invariant with respect to the complex 
G-isotropy group Gp^ = Q. 

Proof. — If U is a sufficiently small open neighborhood of the identity in Q, 
then U.[po] C D. If u{p) ^ [po] for some u e U and some p G [po], then 
there is either a cycle Ci which contains po and not u{p) or a cycle C2 which 
contains u{p) and not pQ. In the first case, contrary to p G [po] the cycle u~^{Ci) 
contains po but not p. In the second case, C2 is not of the form m(C) for some 
cycle containing po, because pQ ^ C2. Thus u~^(C2) is a cycle containing p but 
not Pq and as above this is contrary to p G [po]. Consequently u{p) G [po] for 
all M G L7 and p G [po] and, since U generates Q as a group, it follows that [po] 
is Q-invariant. □ 

Therefore the stabilizer Q of [po] in G is a parabolic subgroup of G which de- 
fines a holomorphic fibration G/Q = Z Z := G / Q. Since Q D Iq and Iq 
acts transitively on [po]/ it follows that Q/Q = [po\- This information can be 
summarized as follows. 

Proposition 3.8. — The restriction of the holomorphic fibration n : G/Q = Z ^ 
Z = G/Q to D is the reduction ttq : Gq/Hq = D ^ D = G/ Iq of D by cycle 
separation equivalence. For p ^ D the fibers of the two maps agree: Ti~^{n{p)) = 
7iQ^{no{p)). In particular, D is afiag domain in Z. 

Let us say that that cycles separate the points of a flag domain D if D = D. The 
following shows that one actually gains something by reducing to D. 

Proposition 3.9. — For every flag domain D the cycles separate the points of the 
base D of the cycle separation equivalence relation. 

The next lemma is the essential ingredient for the proof of this Proposition. For 
this we let ^ := dim (Co) and denote by n* the the map from the full space of q- 
cycles in Z to the full space of ty-cycles in Z which is induced from the fibration 
7T : Z ^> Z. 

Lemma 3.10. — The restriction of n* to the irreducible component Cq(Z) is a bi- 
holomorphic map tt* : Cq{Z) — )■ Cq{Z). In particular, tCq : Cq{D) Cq{D) is also 
biholomorphic. 
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Proof. — The map n* is holomorphic and injective and therefore its image lies 
in some irreducible component of the full cycle space of Z. Since tt*{Co) = 
Co G Cq{Z), its image is in Cq{Z). Now consider the map tt* from supports of 
cycles in Q(Z) to compact analytic subsets in Z. Let us say that a cycle C G 
Cq{Z) is TT-saturated if its image is ^-dimensional. Clearly the set of saturated 
cycles is closed. To see that it is also open, suppose C is saturated, but that there 
is a sequence C„ of nonsaturated cycles converging to C. It follows that 7r(C„) 
converges to n{C) in the Hausdorff topology. But this is impossible, because 
dim(7r(C„)) > dim(7r(C)). Since Co is saturated, it follows that every C G 
Cq{Z) is saturated and therefore tt* maps Cq{Z) to an irreducible subvariety of 
the full space of ^-cycles in Z. Since n^,{Co) = Cq, this subvariety is contained 
in C^(Z). Therefore n* is invertible with inverse n^,. The biholomorphicity of 
tTq follows immediately. □ 

Proof of Proposition 13.91 Observe that if p G D, then by the Lemma the family 
of cycles containing p is mapped biholomorphically by tTq to J-'p for any 
p G (p). If p and q are different points in D and p and q are corresponding 
points in D, then 7^ Tq. Thus 7^ Tq and consequently p and q are 
separated by some cycle in D. □ 

Since Cq{D) is Kobayashi-hyperbolic with Auto(Q(D)) being a Lie group act- 
ing smoothly by holomorphic transformations, the following formal observa- 
tion is important for our study of Aut(D). 

Proposition 3.11. — If cycles separate points in D, then the canonically defined ho- 
momorphism Auto{D) —> Autc)(Cq(D)) is injective. 

Proof. — If g G Aut(D) acts as the identity on the cycle space, \hervg{Tq) = Tq 
and consequently g{Tq) = Tq for all q ^ D. Recall that [p] is the intersection 
of such sets and [p] = {p} in the case where cycles separate points. Thus if 
cycles separate points and g acts trivially on the cycle space, it follows that g 
acts trivially on D. □ 

As a consequence it follows that for an arbitrary flag domain D the ineffec- 
tivity of the Autc)(D)-action on Cq{D) is the same as the ineffectivity of its 
action on the base of the reduction D ^ D defined by cycle separation. In the 
next section we show that, unless D is holomorphically convex with nontrivial 
Remmert reduction, the latter ineffectivity is finite. 



14 



ALAN HUCKLEBERRY 



4. Ineffectivity of the action on the cycle space 

In this section it will be assumed that the flag domain at hand is cycle con- 
nected, or equivalently that D it is not holomorphically convex. Recall that 
this is the same as assuming that 0{D) = C. As above D — > D denotes the 
reduction by cycle separation. 

Our goal here is to prove that the ineffectivity I of the Autc)(D) -action on the 
cycle space Cq{D) is a finite (perhaps trivial) group. For this we must slightly 
refine our considerations for the reduction by cycle separation. This is done as 
follows. 

The reduction by cycle separation D — )■ D is constructed using all cycles in 
Cq{D), because for the purposes of analyzing Auto(D) we require it to be 
Auto (D)-equivariant. On the other hand, by doing this we loose control of the 
geometric nature of the cycles, because they are not necessarily orbits. Having 
paid this price, we now reduce D by cycle seperation using only those cycles 
from Mf). Thus we obtain a sequence 

D ^ D ^ Dres 

of Gg-homogeneous fibrations. The notation Dres refers to the fact that we have 
restricted to the smaller set of cycles Mq. Note that since is naturally 
isomorphic to Md, we my view D — )■ Dres as reduction by cycle separation 
using Md- 

The methods which are used for proving the basic properties of D — )■ D may 
be used for D — )• Dyes- For example, the sequence of homogeneous fibrations 
above are saturated restrictions of fibrations 

In this case, however, we only know that Ttres '• D — )■ Dres defines a Gq- 
equivariant biholomorphic map from Mures Md- 

Now we come to the main point: Although D — )■ Dres is not necessarily 
Auto (D)-equivariant, it is /-invariant! This is just due to the fact that D — ^ Dres 
can be defined by continuing the J-invariant map D^D to D^D^ Dres- 
Using this we will show that J is a (finite-dimensional) Lie group acting 
smoothly on D as a group of holomorphic transformations. 

The Lie group structure of /. — Our work here makes strong use of the 
proposition below. It will be applied to the associated fibration tt : C — )■ Cres 
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of a cycle C in A^d to Cres £ -^Dre, which is defined by the restriction of the 
reduction D — > Dres- Note that if C = g{Co), then C and Cres are homogeneous 
with respect to G := gKg~^ and n is G-equivariant. 

Proposition 4.1. — Let n : G/Qi = Zi — > Z2 = G/Q2 be a homogeneous fi- 
bration of flag manifolds. Let Ii be a Lie group acting effectively on T\ as a closed 
subgroup of Kvl\.q{Z\) which stabilizes every n-fiber n~^{z'2) =: F. Then I\ acts 
effectively on every n-fiber. 

Proof. — We may replace I\ by the (algebraic) subgroup I of the algebraic 
group Aut(5(X) which stabilizes the zr-fibers. Now Aut(5(X)° is semisimple 
and 1° is a product of its simple factors. Thus Zi = A x B, where A is the cor- 
responding product of irreducible factors of Zi. Consequently A is contained 
in every zr-fiber and as a result 1° acts effectively on every zr-fiber as well. Fi- 
nally, since 1 is an algebraic group 1° is semisimple, any normal subgroup F of 
I is finite. Therefore, if F is the ineffectivity of the J-action on a zr-fiber F, at any 
point z G F we may faithfully linearize the F-action on the tangent space Tz^i 
to produce a local submanifold A complemenary to F at z which is F-invariant 
and where F acts faithfully. This is contrary to F stabilizing all zr-fibers. □ 

The following is the main result of this section. 

Theorem 4.2. — IfDis a flag domain which is not holomorphically convex, then the 
ineffectivity I of the Aut{D)-action on the base of the reduction D ^ D defined by 
cycle separation is a Lie group acting smoothly as a group of holomorphic transforma- 
tions on D. 

Before giving the proof, let us note that by the basic result of Bochner and 
Montgomery ( ||BM[ , Theorem 4) it is enough to show that J is a Lie group 
acting continuously on D. Thus we begin by looking a bit carefully at the topo- 
logical properties of L For this let v : X — > A^d be the universal family where 
X = {(z, C) G D X A^d;z G C} and v is the projection on the second factor. 
Again we emphasize that Auto{D) may not act on X but I does, because it 
stabilizes every cycle in Aio- Since v is proper and /-invariant, we may apply 
the theorem of Arzela-Ascoli-Montel to prove the following fact. 

Proposition 4.3. — Every sequence {gn} G I has a subsequence which converges in 
I. 
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Proof. — The application of the theorem of Arzela-Ascoli-Montel shows that 
after passing to a subsequence {gn} converges to g ^ Auto{X). But this im- 
plies that gn\C — > g\C for every C G A^d- Now view g„ as a sequence in 
Auto(D). Given x G C C D the image point gn{x) doesn't depend on C. Thus 
the map g : D — > D is well-defined by g{x) := g\C{x). In other words g 
descends via the map X — ?■ D to an automorphism of D. □ 

Given C G Md we let Rq I ^ Auto{C), g ^ g\C, the restriction map. The 
following is an immediate consequence of the the above compactness state- 
ment and the continuity of Rq- 

Corollary 4.4. — The restriction map Rq : I ^ Auto{C) is a closed map. 

Proof of Theorem \42\ Consider the J-invariant fibration n : D ^ Dyes defined by 
cycle separation with cycles in M.^. If C = g(Co) G M.Dr then the restriction 
TTc of TT to C maps C to a cycle Cyes in Dyes' ^'^^ cycles are homogeneous 
with respect to G := gt^g~^ and ttq '■ C ^ Cres is G-equivariant. Let 7c be the 
image of I in Aut0(C) and Ic{F) the image of 7c iri the automorphism group 
of a given zic-fiber F. The above Propostion shows that the map 7c — ^ 7c (f) 
defined by restriction is an isomorphism. 

Since D is not holomorphically convex, we know that it is cycle connected 
with respect to cycles in Ado. Thus Dres is cycle connected with respect to 
cycles in A^g^^.- Given two points p and q in Dres we connect them with a 
chain C^g^ U . . . U OJ^g with intersection points pj so that pQ = p and p„ = q. 
Let Fj be the zr-fibers over the pj. The above argument shows that if g G 7 
is in the kernel of the restriction map 7 I^j, then it is in the kernel of the 
restriction map 7 I(^j{Fj). Again applying the above argument, this implies 
that g is in the kernel of 7 — > Iqi+i. Using the fact that D is cycle connected, 
this shows that if g is in the kernel of the restriction map 7 — )■ 7c (F) for some 
cycle C and some fiber F, then g = Id, i.e., the homomorphism J^c • ^ ~^ ^c{F) 
is an injective continuous homomorphism onto a subgroup of the Lie group 
Autc)(F). It follows from the above Lemma that J^c is a homeomorphism onto 
a closed subgroup 7c (F) of Autc)(F). Since 7c (F) is a Lie group, it follows 
follows that 7 is a Lie group (see Theorem 1.1 of Chapter VIII in [H]). Since it 
is a (closed) subgroup Autc)(D), the action of 7 on D is continuous. □ 

We have now shown that ineffectivity 7 is a Lie group. It is normalized by Gq 
and, since both 7 and Gq are subgroups of the topological group Autc)(D), the 
action of Gq on 7 by conjugation is continuous. By the results in Chapter IX 
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of IHl, in particular Theorem 3.1, the semidirect product J x Go is a Lie group 
which is acting continuously on D. The following is therefore a consequence 
of the Theorem of Bochner and Montgomery ( |BM| ). 

Corollary 4.5. — The Lie group 7 x Gq acts smoothly as a group of holomorphic 
transformations ofD. 



Finiteness of the ineff activity. — After all this discussion about the ineffec- 
tivity I we are now in a position to show that it is essentially nonexistent: Ex- 
cept when D is holomorphically convex with a nontrivial Remmert reduction, 
I is finite! The following is the remaining tool needed for proving this. 

Proposition 4.6. — IfGo is a connected Lie group acting effectively on D as a group 
of holomorphic transformations and Gq D Gq, then the action Gq x D ^ D extends 
to a smooth action Gq x Z ^ Z by holomorphic transformations. 

Proof. — Let n := dimc(D) and consider the space V of sections of the anti- 
canonical bundle of D spanned by elements of the form A . . . A ^„ where the 
are arbitrary holomorphic vector fields defined by the local G-action on D. 
Define the g-anticanonical map cp : D ^ F(V*) by p {s G V;s{p) = 0}. 
Since Gq acts on V and D is homogeneous, ^ is a Go-equivariant holomorphic 
map (see HHOI for other basic properties of this map). 



Let Pq be the neutral point in D and Zq := cp{pQ) its image in P(V'*). Now 
D is embedded in the G-flag manifold Z whose anticanonical bundle is very 
ample. In fact the g-anticanonical map extends to Z = G/Q as the Tits fibration 
G/Q G/N, where N is the normalizer of Q in G. Since N = Q (Borel's 
Normalizer Theorem), the g-anticanoncal map of D is an embedding. Since 
the vector space V which is defined by limiting the to fields defined by g is 
contained in V, it is therefore immediate that (p is an embedding. 

Thus we may regard D as Go-orbit Gq.zq in P(V'*) and define G to be the small- 
est complex Lie subgroup containing the group Gq in GL(V'). Hence D can be 
regarded as the open Go-orbit Go.zo in the complex orbit G.Zo. 

After this lengthy background, we consider the complex orbit G.zo which is an 
open subset in G.Zo. Since cp is an embedding, the isotropy algebra Qzq is just 
the Lie algebra q. Thus the Lie algebra of Gz^ is q and therefore G^g = Q, in 
particular G.Zo = G/Q = Z is compact. Since G.Zo is open in G.Zo and both 
groups are connected, it follows that G.Zo = G.Zo = Z as desired. □ 
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Theorem 4.7. — Unless D is holomorphically convex with nontrivial Remmert re- 
duction, the ineffectivity I of the action o/Aut(p(D) on the cycle space Cq{p) is finite. 

Proof. — Note that the connected component 1° is normalized by Go and 
Corollary 14.51 implies that Gq := 1° xi Gq is a connected Lie group which acts 
smoothly as a group of holomorphic transformations on D. The ineffectivity 
of this action is contained in the center of Gq and is therefore discrete. By def- 
inition Go acts on Z and it follows from the above Proposition that Gq acts on 
Z. By definition the complexification G acts transitively on Z and therefore the 
smallest complex subgroup G of A\x\.o{Z) which contains Gq also acts transi- 
tively. 

We claim that G properly contains G. To see this note that since G acts transi- 
tively on Z, it is semisimple. Thus the Lie algebra i is semisimple and go is the 
direct sum i qq. As a result q is not simple. Thus if g = g, it follows that g 
is not simple and therefore go is (abstractly) a simple complex Lie algebra em- 
bedded antiholomorphically in its complexification g = go © go- But if g = g, 
then the image of go in g centralizes i. On the other hand this image has trivial 
centralizer. Thus g properly contains g. 

As a result we are in the situation where two complex Lie groups G and G act 
transitively on Z. Since G properly contains G, it follows that Z = G/Q where 
Q is a maximal parabolic subgroup ( i01i[02l '). But this situation arises when 
D ^ D is a nontrivial reduction by cycle separation. Since this reduction is 
the restriction of a fibration Z ^ Z of the ambient flag manifold, Q is not a 
maximal parabolic subgroup of G. Thus i is trivial and therefore I is discrete. 
But since it is normalized, and therefore centralized by Go, it is finite. □ 

5. Lie group properties 

Let us begin by stating the main result of this section. For this we recall that if 
a flag domain D is not cycle connected, then it is holomorphically convex and 
the reduction D — ^ D by cycle connectivity equivalence is its Remmert reduc- 
tion. The base D is a Hermitian symmetric space of noncompact type and is 
therefore realizable as a bounded Stein domain. In particular, if D = D, then 
Auto (D) is a Lie group acting smoothly on D as a group of holomorphic trans- 
formations. If D is not cycle connected and D 7^ D, then Aut0(D) is infinite- 
dimensional, but nevertheless easily describable (see l3.4|) . Here we shall prove 
that if D is cycle connected, then Autc)(D) is a Lie group acting smoothly on 
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D as a group of holomorphic transformations. This then completes the proof 
of the following result. 

Theorem 5.1. — IfDis a flag domain with reduction D — )• D &y cycle connectivity, 
then one of the following three cases holds: 

1. D is not cycle connected, in which case it is holomorphically convex. The reduc- 
tion D ^ D is the Remmert reduction ofD has positive-dimensional fiber F and 
positive-dimensional base D which is a Hermitian symmetric space of noncom- 
pact type. The group of automorphisms, which can be described by 

Auto(D) ^ Hol(D, Auto(F)) x Auto(D) , 

is infinite-dimensional. 

2. D is a Hermitian symmetric space of noncompact type, i.e., D = D, and 
Auto{D) is an Lie group acting smoothly on D as a group of holomorphic trans- 
formations. 

3. D is cycle connected, i.e., D is just a point, and Auto{D) is a Lie group acting 
smoothly on D as a group of holomorphic transformations. 

Let us now complete the proof of this theorem. We must only handle the case 
where D is cycle connected or equivalently where 0(D) = C. The proof 
uses the fact that AuioiD) can essentially be identified with a subgroup of 
AutoiCq(D)). The following is a first step in this direction. 

Proposition 5.2. — IfD is any flag domain, then the canonically deflned continuous 
homomorphism 

I : AuioiD) ^ AuioiCciD)) 

is a closed map. 

Proof. — Let X C D x Cq{D) be the universal family of cycles with its pro- 
jections }i : X ^ D and v : X — )• Cq{D). Suppose f is a closed subset of 
Autc)(D) and that gn — i(hn) defines a sequence in i(f ) which converges to 
g e Autc){Cq{D)). Observe that the gn act on X and denote them there by gn- 
Since v : X — > C^(D) is proper and gn — > g, it follows that the sequence {gn} is 
equibounded. It therefore follows from the theorem of Arzela-Ascoli that after 
going to a subsequence gnp. g, where g is a lift of g to X. Now the gn are lifts 
of the hn and since gn — )• g, the automorphism g ofX descends to an automor- 
phism h oiD with hn — )■ h. Since F is closed, /z e F and, since / is continuous, 
i(h) — g. Thus /(F) is likewise closed. □ 
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Corollary 5.3. — If cycles separate points in D, then 

i: Auto (D) ^ Auto (Q(D)) 
zs an injective homeomorphism onto a closed subgroup of Auto {Cq{D) ). 

Proof. — It remains to prove the injectivity. However, this is the content of 
Proposition I3.11[ □ 

Using the Theorem of Bochner and Montgomery ( [|BM | | , Theorem 4) the above 
corollary shows that Theorem 15. 1 1 holds in the case where D = D. In the case 
where the fiber of the reduction D — ^ D is positive-dimensional we make use 
of the following result. 

Proposition 5.4. — The continuous homomorphism 

TT* : Auto(D) Auto(D) 
which is induced by cycle separation n : D ^ D isa closed map. 

Proof. — Recall that n induces a biholomorphic map Cq{D) Cq{D). If f is a 
closed subset of Auto (D) aridg„ = n^Qin) G 7r*(f) defines a sequence which 
converges to g G Auto(D), then the associated sequence of automorphisms 
of Cq{D) converges to the automorphism of the cycle space associated to g. 
Thus the sequence {z(/z„)} on the cycle space Cq{D) which is associated to 
{hn} converges and the result follows from Proposition 15.21 □ 

As a result we know that the image of tt* is a closed subgroup of Auto(D) and 
is therefore a Lie subgroup. Recall that the Ker(7r*) = I is finite. Thus the map 
TT^ : Auto(D) — > Auto(D) is a topological covering map of a Lie group. As 
a result Auto(C>) is homeomorphic to a Lie group and again by the Theorem 
of Bochner and Montgomery it follows that Auto(D) is a Lie group acting 
smoothly on D as a group of holomorphic transformations. This completes the 
proof of Theorem 15. 1[ 

6. Detailed description of Auto (D) 

Recall that our initial setting is that of a simple real form Gq of a complex 
semisimple group G acting on a G-flag manifold Z = G/Q. Except in the case 
where D holomorphically convex with nontrivial Remmert reduction, where 
Auto(D) is a certain precisely described infinite-dimensional group, Auto(D) 
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is a Lie group acting smoothly on D by holomorphic transformations. In this 
section we restrict to the latter case and give a more detailed description of 
Auto{D). Here is a first step in that direction. 

Proposition 6.1. — If Q is not maximal, i.e., if the Betti number &2(Z) is at least 
two, then Autc)(D)° = Gq. 

Proof. — If Autc)(D)° =: Gq, then by Proposition 14.61 the action of Gq extends 
to a smooth action Gq x Z ^ Z of holomorphic transformations on Z. Since 
Go D Go, the smallest complex Lie group G in Aut0(Z) which contains Go 
also acts transitively on Z. As we observed in the proof of Theorem 14. 7[ if Go 
properly contains Gq, then G properly contains G. Thus it follows from the 
work of Onishchik (|IOll[02l) that Q is maximal. □ 

Corollary 6.2. — IfGis defined to be Auto{Z)° and D is not holomorphically con- 
vex with nontrivial Remmert reduction, then Autc)(D)° = Gq. 

Exceptional cases. — If G is not the connected component G of the full au- 
tomorphism group Autc)(Z) then, as we just noted, Q is a maximal parabolic 
subgroup of G and we are in one of the situations classified by Onishchik (see 
noil 1021 ). There are two series of flag manifolds Z where this is possible and 
one additional isolated example: 

- Odd dimensional projective spaces where G = SL2„(C) and G = Spjj(C). 

- The space of isotropic n-planes with respect to the standard complex bi- 
linear form on C^" where G = S02n(C) and G = S02n-i(C). 

- The 5-dimensional quadric where G = S07(C) and G = G2. 

Example. Let y be a complex vector space and equip W = V ®V* with its 
standard symplectic form which is defined by 



Let {ei, . . . , e„} be a basis of V. Translating to the numerical space, for z, w G 
C^" it follows that (X){z, w) = z^Jw where 



co{v + (p,v' + cp') = (p'{v) - (p{v) . 




Define the Hermitian form h on C 



by 



h{z,w) 



-z Jw . 
2 
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It is of signature (n, n). For example a maximal negative (resp. positive) space 
is spanned by vectors of the form (a, ia) (resp. (a, —ia)). 

Let G :— SLc(W) and G := Sp^(W,a;). Define Gq to be the subgroup of h- 
isometries in G and Go = Go n G. Observe that Go = SU(n, n) and Go = 
Sp2„(]R). 

One directly checks that Go has exactly three orbits in Z = P(W), namely the 
spaces D+ and D~ of positive (resp. negative) (complex) lines and the real 
hypersurface L of isotropic lines. 

To determine the orbit structure of the Go-action is convenient to use Matsuki 
duality. For this we choose the maximal compact subgroup Kq of Gq to be the 
copy of the unitary group U„ which acts diagonally on W, i.e., k(v + cp) = 
k{v) + k{(p). The complexification ]<C of i^o in G is just GL„(C). The duality 
theorem states that there is a natural bijective correspondence }i between the 
JC-orbits and the Go-orbits in Z. This sends a Go-orbit O to the K-orbit }i{0) — 
K.p of any point p ^ O with Kg.p the minimal Ko-orbit in O. Furthermore, 
}i(0) n O is just the minimal Ko-orbit KQ.p. 

We mention this "duality", because it is often easier to understand the K-orbit 
structure than it is to understand the Go-orbit structure. In this case one checks 
directly that K has four orbits in Z, namely the closed orbits P(^) and ¥{V*), 
a 1-codimesional orbit with two ends whose closure Y consists of the orbit 
together with the projective subspaces P(V) and P(y*), and the complement 
Z\Y which is the unique open JC-orbit. Duality implies that the closed K- 
orbits are the base cycles in the open Go-orbits. Thus Go has two open orbits 
which are of course contained in the two open Go-orbits D"*" and D~. The 
smaller group Gq stabilizes the closed Go-orbit E. There the real points Eq are 
Go-invariant. Thus Go has at least two orbits in L. But altogther it has only 
four orbits and therefore we have accounted for all of them: D+, D~, Eq and 
the complement E \ Eq. In particular, D+ and D~ are exceptional in the sense 
that the smaller real form Gq acts transitively. □ 

It would be of interest to have a complete list of all such exceptional cases, in 
particular to determine if and when a given Go-flag domain is properly con- 
tained in the corresponding Go-flag domain. Arguing as above one could pos- 
sibly compile such a list. This would, however, seem to require a somewhat in- 
volved case-by-case discussion which would not be appropriate for the present 
paper. 
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